All Questions
Tagged with anomaly-detectionautoencoder
32 questions
0votes
0answers
40views
Does Increasing Dimensionality Before Compression Make Sense for Anomaly Detection with Autoencoders?
Given a dataset $X$ of shape $(n, p)$ such that $n \gg 1$ and $p \approx 10$, I would like to train an autoencoder to solve an anomaly detection problem. I did some experiments considering a classical ...
0votes
0answers
71views
Choosing Between Autoencoder with OC-SVM and Reconstruction Error for Anomaly Detection: Training Strategies and Considerations
I plan to use an autoencoder and One-class SVM (OC SVM) for anomaly detection. So there are 2 strategies: train the autoencoder, and use the encoder output (as reduced dimension) to train an OC SVM ...
- 521
2votes
2answers
282views
Autoencoders are fitting anomalies too good
I have a set of ~ 5000 greyscale images with resolution of 64x128. I want to do an unsupervised anomaly detection. As a first try, I chose convolutional autoencoders (AE) and trained an AE model. I ...
0votes
1answer
233views
Constructing an LSTM autoencoder for variable-lentgth sequences
I would like to construct an LSTM autoencoder model for sequence anomaly detection where the sequences can be varying in length. I understand based on this answer that padding and masking can be used ...
- 133
1vote
0answers
124views
Threshold for auto encoder anomaly detection
I have fitted an auto encoder on 25-dimensional time series data hoping to be able to detect anomalies. training set is 100k observations, testset for threshold setting is 10k observations. all ...
- 21
2votes
1answer
749views
Early anomaly detection / Failure prediction on time series
My problem here is that I want to predict failures in advance with respect to their occurrence. I have sensors mounted on my machine and with a certain frequency, they send data to my database. ...
- 45
0votes
1answer
153views
An autoencoder setup for anomaly detection
I am doing anomaly detection using machine learning. i have tried different models like isolation forest, SVM and KNN. The maximum accuracy that I can get from each of them is $80\%$ accordind to my ...
- 11
0votes
0answers
36views
How to detect anomalies?
I have timeseries data with one value per day for a year. (there is one column with temperature data). I am using autoencoders to train a reconstruction model with mse loss. Firstly, I normalized the ...
1vote
1answer
772views
Training data for anomaly detection using LSTM Autoencoder
I am building an time-series anomaly detection engine using LSTM autoencoder. I read this article where the author suggests to train the model on clean data only in response to a comment. However, in ...
0votes
1answer
368views
Incremental learning on Autoencoder for anomaly detection
I want to incrementally train my pre-trained autoencoder model on data being received every minute. Based on this thread, successive calls to model.fit will incrementally train the model. However, the ...
- 11
0votes
1answer
766views
how to set threshold value by looking at loss distribution in anomaly detection task
I am following this tutorial https://towardsdatascience.com/lstm-autoencoder-for-anomaly-detection-e1f4f2ee7ccf to use LSTM autoencoder to detect anomalies in my unsupervised dataset. they plotted ...
- 171
1vote
2answers
267views
Autenocoder and anomaly detection task
I'm trying to create an autoencoder for the anomaly detection task, but I'm noticing that even if it performs very well on the training set, it starts to stop recreating half of the test set. I tried ...
- 53
0votes
1answer
412views
Understanding time series anomaly detection using Autoencoder
I'm studying how to detect anomalies in the time series using an Autoeconder. In particular, I'm following the guide posted in the Keras website, but I don't understand why they are creating and how ...
- 53
1vote
1answer
504views
Is window based sequencing a good idea to obtain more training data for LSTMs?
I am trying to do an unsupervised autoencoder based outlier detection for time series using LSTMs. Here, there are multiple time series, and an entire series is to be considered as an outlier. However,...
- 41
1vote
2answers
1kviews
Cross-Validation in Anomaly Detection with Labelled Data
I am working on a project where I train anomaly detection algorithms Isolation Forest and Auto-Encoder. My data is labelled so I have the ground truth but the nature of the problem requires ...
